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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address « 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )[x] Responsive to communication(s) filed on 10 November 2000 . 
2a)D This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) K Claim(s) 1-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |3 Claim(s) 1-28 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 10 November 2000 is/are: a)D accepted or b)l3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

1 1) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) [x] Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)l3 All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. ^ Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 

1 ) £3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) \Z\ Notice of Informal Patent Application (PTO-152) 

3) (3 Information Disclosure Statement(s) (PTO-1449) Paper No(s) 3. 6) □ Other: 
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DETAILED ACTION 

1. This application is a 371 of PCT/GB99/01506 internationally filed on 05/12/1999. 

Information Disclosure Statement 
1. The information disclosure statement filed January 22, 2001 fails to comply with the 
provisions of 37 CFR 1.97, 1.98 and MPEP § 609 because reference 1003022A3 and 2-595-940 
did not have English translation and therefore have not been considered. It has been placed in 
the application file, but the information referred to therein has not been considered as to the 
merits. Applicant is advised that the date of any re-submission of any item of information 
contained in this information disclosure statement or the submission of any missing element(s) 
will be the date of submission for purposes of determining compliance with the requirements 
based on the time of filing the statement, including all certification requirements for statements 
under 37 CFR 1 .97(e). See MPEP § 609 If C(l). 

Drawings 

1. The drawings are objected to because the box elements 80, 82, and 84 in Figure 8 need to 
be labeled in accordance with 37 C.F.R. § 1.83(a) as stated infra. A proposed drawing correction 
or corrected drawings are required in reply to the Office action to avoid abandonment of the 
application. The objection to the drawings will not be held in abeyance. 

Specifically, 37 C.F.R. § 1.83(a) states that "the drawing in a nonprovisional application 
must show every feature of the invention specified in the claims. However, conventional features 
disclosed in the description and claims, where their detailed illustration is not essential for a 
proper understanding of the invention, should be illustrated in the drawing in the form of a 
graphical drawing symbol or a labeled representation (e.g., a labeled rectangular box)." 
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2. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(4) because 
reference character "(a)" has been used to designate both Fig. 7 (a) and Fig. 7(b). See page 6/9 of 
the drawings. Specification discloses Fig. 7b on page 15 2 nd paragraph lines 7-8. A proposed 
drawing correction or corrected drawings are required in reply to the Office action to avoid 
abandonment of the application. The objection to the drawings will not be held in abeyance. 

Specification 

3. This application does not contain an abstract of the disclosure as required by 37 
CFR 1.72(b). An abstract on a separate sheet is required. 

4. The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. 

5. The disclosure is objected to because of the following informalities: 

/ a. On page 2, 3 paragraph line 2, the word "ands" ought to be "and 
^ b. On page 10, 2 nd paragraph line 2, "figure 10" ought to be "feature 10." 

c. On page 12, 3rd paragraph line 3 "form" ought to be "from." 

d. On page 12, last line, please delete re-occurrence of the word "areas." 

e. On page 23, 3 rd paragraph line 9, the word ""wand" ought to be ""wand."" 
Appropriate correction is required. 

Claim Objections 

6. Claims 5 and 15 are objected to because of the following informalities: 

a. In claim 5 line 3 the word "represent" ought to be "representing " 

b. Claim 5 line 3 "represent the image as a 2D image" needs a comma "," or a semi- 
colon ";" after the word "image." 



Application/Control Number: 09/700,087 Page 4 

Art Unit: 2621 

c. In claim 15 line 2 please delete the underline from the word " optionally " as claim 
15 is an original claim as filed. Note: Underline represents addition in the amended claims. 
Appropriate correction is required. 

Claim Rejections - 35 USC § 112 

7. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

8. Claims 2, 5, 9, 15-18, and 21-22 are rejected under 35 U.S.C. 1 12, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

9. Claim 2 recites the limitation "the distribution 1 ' in line 2 of claim 2. There is insufficient 
antecedent basis for this limitation in the claim. 

10. Claim 5 recites the limitation "the distribution" in lines 5-6 of claim 5. There is 
insufficient antecedent basis for this limitation in the claim. 

1 1 . Claim 9 recites the limitation "the full image" in lines 1-2 of claim 9. There is 
insufficient antecedent basis for this limitation in the claim. 

12. Claim 15 recites the limitation "the centre" in lines 2 and 3 of claim 15. There is 
insufficient antecedent basis for this limitation in the claim. 

13. Claim 15 recites the limitation "the highest spatial resolution" and "the lowest spatial 
resolution" in line 3 and 4, respectively, of claim 15. There is insufficient antecedent basis for 
this limitation in the claim. 

14. Claim 16 recites the limitation "the eye" in line 3 of claim 16. There is insufficient 
antecedent basis for this limitation in the claim. 
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15. Claim 17 recites the limitation "the duration" in line 2 of claim 17. There is insufficient 
antecedent basis for this limitation in the claim. 

16. Claim 18 recites the limitation "the colour" and "the harmonics" in line 1 and 3, 
respectively, of claim 18. There is insufficient antecedent basis for this limitation in the claim. 

17. Claim 21 recites the limitation "the brightness" and "the intensity" in line 1 and 2, 
respectively, of claim 21. There is insufficient antecedent basis for this limitation in the claim. 

1 8. Claim 22 recites the limitation "the relative depth" in line 3 of claim 22. There is 
insufficient antecedent basis for this limitation in the claim. 

Claim Rejections - 35 USC § 102 

19. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

20. Claims 1-19 and 21-28 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Meijer (USPN 5,097,326). 

With regard to claim 1 Meijer discloses a method enabling a person to visualize images 
(See, Fig. 2) comprising the steps of: encoding spatial information (encoding spatial information 
by converting into an acoustical representation. See, col. 3 lines 33-34 and lines 60-63) relating 
to a feature or features contained within an image (i.e., the visual images taken by the camera. 
See, col. 3 lines 30-34.) into the form of one or more musical sequences (See, col. 3 lines 60- 
63)(image-to-sound conversion as seen in Fig. 2 and see col. 4 lines 35-61. Note: Meijer 
discloses the signal being a music or a speech at col. 7 lines 58-61); and playing the musical 
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sequence or sequence to the person (i.e., playing (outputting) the music sequence to the 
headphones thru which the person listens. See, col. 4 lines 50-54 and col. 7 lines 41-46). 

With regard to claim 2 Meijer discloses spatial information encoded by selecting a note 
or chord (note: Generally, music inherently includes notes and/or cords) dependent on (the 
distribution) of the feature or features along an axis (image scanning in vertical lines. See, col. 3 
lines 58-60 and col. 7 lines 31-32). 

With regard to claim 3 Meijer discloses the image comprising a letter or a number 
(Meijer discloses broadly "Images 20" that is visual at col. 4 lines 35-37 and that includes a letter 
or a number). 

With regard to claim 4 Meijer discloses the image comprising a person's environment 
(Meijer discloses broadly "Images 20" that is visual at col. 4 lines 35-37 and that includes a letter 
or a number). 

With regard to claim 5 Meijer discloses spatial resolution information by (represent) the 
image as a 2D image (the image 9 in Fig. 1 has the representation of X- Y-axis, hence 2D image) 
and forming one or more musical sequence, each comprising a series of notes or chords (as 
described in claim 1), in which i) each note or chord is selected dependent upon the distribution 
of the feature or features along a portion of the 2D image (as described in claim 2 and visual 
image is represented in 64 columns and 64 rows at col. 3 lines 36-53) and ii) different notes or 
chords in a sequence correspond to different portions of the 2D image (See, col. 8 lines 66-68 to 
col. 9 lines 1-2). 

With regard to claim 6 Meijer discloses the 2D image, which is divided into a matrix of 
pixels (as seen in Fig. 1 and col. 3 lines 54-60), and i) each note or chord is selected dependent 
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upon the distribution of the feature or features along a column (or rows) of pixels (as described 
in claims 1 and 2 and visual image is represented in 64 columns and 64 rows at col. 3 lines 36- 
53) and ii) different notes or chords in a sequence correspond to the distribution of the feature or 
features along different columns (or rows) of pixels (see, col. 8 lines 66-68 to col. 9 lines 1-2). 

With regard to claim 7 Meijer discloses a different note, which is associated with each 
pixel along a column (pixel in a column as seen in Fig. 1. See, col. 3 lines 54-58) and, if a 
feature recognized as being present in a pixel, the note corresponding to that pixel comprises part 
of the musical sequence (the signal representation 12 of the pixel 10 is summed and converted 
into an acoustical signal 16 as seen in Fig. 1. See, col. 4 lines 3-11). 

With regard to claim 8 Meijer discloses enabling a person to visualize moving features 
comprising the step of playing a plurality of musical sequences corresponding to different 
positions and/or orientations of the moving features (Meijer discloses a video camera taking a 
sequence of image of an moving object and converting this to an acoustical signal, col. 7 lines 
13-14 at different positions and/or orientation at col. 3 lines 34-36). 

With regard to claim 9 Meijer discloses a subset of the full image that is encoded into the 
musical sequence or sequences (see a subset (each row and column) being encoded in an 
acoustical signal from 10 to 12 in Fig. 1) 

With regard to claim 10 Meijer discloses predetermined features that are extracted from 
the image, and said predetermined features are encoded into a musical sequence or sequences (as 
described above in claim 9, each features in each row and column, are predetermined and 
converted into an acoustical signal before summing up to obtain the entire image in an acoustical 
signal as seen in Fig. 1). 
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With regard to claim 11 Meijer discloses a feature being simplified by encoding a portion 
of the feature as a musical sequence (each feature 10 is being simplified at 12 thru 1 1 as seen in 
Fig. 1). 

With regard to claim 12 Meijer discloses the feature that is encoded by encoding 
different portions of the features as different musical sequences (See, col 8 lines 66-68 to col. 9 
lines 1-2). 

With regard to claim 13 Meijer discloses the image being encoded into the form of a 
plurality of musical sequences, which are played to the person as a melody (i.e., arrangements of 
sounds) (musical sequence of each feature 12 in Fig. 1 are being converted in an acoustical 
signal and played to the person thru a headphones. See, col. 4 lines 3-6 and lines 50-52). 

With regard to claim 14 Meijer discloses the image being encoded as a plurality of 
musical sequence (as described above), each corresponding to different spatial resolutions 
(different frequencies give different spatial resolutions. See, col. 6 lines 63-67). 

With regard to claim 15 Meijer discloses the image is being divided into two or more 
concentric zones (the image is being divided as seen at 9 in Fig. 1 having a common center zone 
with a maximum frequency at col. 8 lines 59-66) the zone at the center of the image being 
encoded at the highest spatial resolution and the zone furthest from the center of the image being 
encoded at the lowest spatial resolution (See, col. 7 lines 35-39). 

With regard to claim 16 Meijer discloses a feature or features are visualized by obtaining 
a plurality of images in a saccadic-like (i.e., rapid, ballistic movement) series of movements, 
functionally equivalent to a scan path that the eye follows when it examines an object (when a 
sequence of video frames captured by a video camera, col. 7 lines 13-14, are converted to an 
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acoustical signals. These signals are summed to obtain a one full image as described in Fig. 1 
and respective portion in the specification. These frames and signals are equivalent to the path 
that the eye would follow when examining an object). 

With regard to claim 17 Meijer discloses the spatial resolution corresponding to a 
musical sequence being indicated by the duration of the notes and chords in the sequence (see, 
col. 6 lines 15-20). 

With regard to claim 18 Meijer discloses (the colour of) the feature or features is encoded 
by producing a musical sequence or sequences which comprises a plurality of different sets of 
waveforms corresponding to the harmonics of different sound instruments mixed in variable 
rations (See, col. 5 lines 51-60 and col. 6 lines 8-14). 

With regard to claim 19 Meijer discloses three waveforms being mixed in variable ratios 
(more than one waveform disclosed at col. 8 lines 17-22). 

With regard to claim 21 Meijer discloses the brightness of the feature or features being 
encoded by varying the intensity of the musical sequence or sequences (See, col. 7 lines 25-26 
and lines 49-57). 

With regard to claim 22 Meijer discloses the image motion cues are translated into 
equivalent sound patterns, which are utilized to segregate foreground features from background 
in 3D and produce information on the relative depth between features (See, col. 4 lines 14-21 and 
the 3D image at col. 7 lines 8-12). 

With regard to claim 23 Meijer discloses a device enabling a person to visualize images 
(Fig. 2) comprising: imaging means (image means 22, 24 and 20) for obtaining images of a 
feature or features; encoding means (image processing unit 26 including imaging means) for 
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encoding spatial information relating to the feature or features according to any of claims 1 to 22; 
and playing means (sound generating means 34) for playing the musical sequence or sequences 
to the person. See col. 4 lines 35-61 for detail information on the device. 

With regard to claim 24 Meijer discloses a video camera at col. 7 lines 13-14. 

With regard to claim 25 Meijer discloses a charge coupled detector at col. 13 lines 5-9. 

With regard to claim 26 Meijer discloses the encoding means comprising a 
microprocessor at col. 10 lines 27-37. Note: image processing unit shown in detail here is a 
microprocessor. 

With regard to claim 27 Meijer discloses the playing means comprising an ear-piece (i.e., 
headphones) at col. 6 lines 50-51. 

With regard to claim 28 Meijer discloses an imaging means being a portable device at 
col. 2 lines 27-30. 

Claim Rejections - 35 USC § 103 



21 . The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

22. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Meijer in view of 
Dallas, Jr. et al. (USPN 4,378,569) (hereinafter, "Dallas"). 

With regard to claim 20 Meijer discloses a method enabling a person to visualize images 
as disclosed in claim 1 . Meijer also discloses sampling a single waveform at col. 5 lines 51-56. 
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Meijer does not expressly disclose the three waveforms being produced by filtering a master 
waveform between different frequency ranges. Dallas discloses the three waveforms being 
produced by filtering a master waveform between different frequency ranges (col. 6 lines 30-38). 
Meijer and Dallas are combinable because they are from the same field of endeavor, i.e., 
converting an image to sound representation. At the time of the invention, it would have been 
obvious to a person of ordinary skill in the art to combine the teaching of Dallas with Meijer. 
The motivation for doing so is that in order to have a three different waveform representing a 
single signal one needs to filter the mast waveform thru different filters in order to obtain three 
unique waveforms. Therefore, it would have been obvious to combine Dallas with Meijer to 
obtain the invention as specified in claim 20. 

Conclusion 

23. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

USPN 3,800,082 - Auditory display for the blind. 

USPN 6,198,395 - Sensor for slight impaired individuals. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Shefali D Patel whose telephone number is 703-306-4182. The 
examiner can normally be reached on M-F 8:30am - 5:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H Boudreau can be reached on 703-305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 
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Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-306-0377. 



Shefali D Patel 



Examiner 

PRIMARY EXAMINER ^ Unft 262 { 



September 22, 2003 



